Skip to content

Conversation

@forsyth2
Copy link
Collaborator

@forsyth2 forsyth2 commented Apr 2, 2025

Summary

This PR is specifically to speed up analysis of existing image checker tests. It will likely replace #685 and #695 completely, but ultimately #520/#698 will be needed to 1) automate the pre-test runs of zppy itself and 2) test many more cfg files than the existing ones tested by the image checker.

Objectives:

Issue resolution:

Select one: This pull request is...

  • a bug fix: increment the patch version
  • a small improvement: increment the minor version
  • a new feature: increment the minor version [Testing only]
  • an incompatible (non-backwards compatible) API change: increment the major version

Small Change

  • To merge, I will use "Squash and merge". That is, this change should be a single commit.
  • Logic: I have visually inspected the entire pull request myself.
  • Pre-commit checks: All the pre-commits checks have passed.

@forsyth2 forsyth2 added the Testing Files in `tests` modified label Apr 2, 2025
@forsyth2 forsyth2 self-assigned this Apr 2, 2025
@forsyth2 forsyth2 force-pushed the improved-image-checker branch 3 times, most recently from 667eec9 to 0828fb4 Compare April 2, 2025 01:24
@forsyth2 forsyth2 force-pushed the improved-image-checker branch from 0828fb4 to 4e3d5b3 Compare April 2, 2025 03:37
@forsyth2
Copy link
Collaborator Author

forsyth2 commented Apr 2, 2025

Using the code as of 4e3d5b3, this is the min_summary.md result:

Summary of test results

Diff subdir is where to find the lists of missing/mismatched images, the image diff grid, and the individual diffs.
Note image diff grids can not yet be constructed automatically.

Test name Total images Correct images Missing images Mismatched images Diff subdir
comprehensive_v2_e3sm_diags 2921 2875 0 46 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/e3sm_diags
comprehensive_v2_mpas_analysis 854 854 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/mpas_analysis
comprehensive_v2_global_time_series 6 1 0 5 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/global_time_series
comprehensive_v2_ilamb 482 482 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/ilamb
comprehensive_v3_e3sm_diags 3750 3696 0 54 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v3_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_comprehensive_v3_test_pr699_try6/e3sm_diags
comprehensive_v3_mpas_analysis 854 854 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v3_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_comprehensive_v3_test_pr699_try6/mpas_analysis
comprehensive_v3_global_time_series 15 1 0 14 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v3_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_comprehensive_v3_test_pr699_try6/global_time_series
comprehensive_v3_ilamb 512 512 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v3_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_comprehensive_v3_test_pr699_try6/ilamb
bundles_e3sm_diags 1472 1472 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_bundles_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_bundles_test_pr699_try6/e3sm_diags
bundles_global_time_series 3 1 0 2 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_bundles_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_bundles_test_pr699_try6/global_time_series
bundles_ilamb 241 241 0 0 /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_bundles_www/test_zppy_20250401/v3.LR.historical_0051/image_check_failures_bundles_test_pr699_try6/ilamb

@forsyth2
Copy link
Collaborator Author

forsyth2 commented Apr 2, 2025

To convert to web pages:

/lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/e3sm_diags

would become https://web.lcrc.anl.gov/public/e3sm/diagnostic_output/ac.forsyth2/zppy_weekly_comprehensive_v2_www/test_zppy_20250401/v2.LR.historical_0201/image_check_failures_comprehensive_v2_test_pr699_try6/e3sm_diags/

@forsyth2
Copy link
Collaborator Author

forsyth2 commented Apr 2, 2025

These numbers actually do match the numbers in #634 (comment), so it appears to be working as intended.

Since this PR only touches testing code (i.e., not user-facing code), and I am the primary user for these changes, no further review is necessary. I do hope to make several more testing updates (e.g., #685, #520/#698), but this PR is an important step forward in automating the testing process.

@forsyth2 forsyth2 marked this pull request as ready for review April 2, 2025 03:47
@forsyth2 forsyth2 merged commit 218f55d into main Apr 2, 2025
5 checks passed
@forsyth2 forsyth2 deleted the improved-image-checker branch April 2, 2025 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Files in `tests` modified

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants